|
Jupyter at Bryn Mawr College |
|
|
| Public notebooks: /services/public/dblank / Experiments | |||
Conx¶
Neural network library in Python built on top of Theano.
! pip install conx -U
import conx
conx.__version__
from conx import Network
net = Network(2, 2, 1)
net
net.get_device()
Ok, now we are ready to train a neural network to perform the XOR function.
inputs = [[0, 0],
[0, 1],
[1, 0],
[1, 1]]
def xor(inputs):
a = inputs[0]
b = inputs[1]
return [int((a or b) and not(a and b))]
net.set_inputs(inputs)
net.set_target_function(xor)
net.test()
%%time
net.train()
net.propagate([1, 1])
net.test()
net.reset()
net.test()
%%time
net.train()
net.test()
Generalization¶
Although we only trained on the corners, we can see what
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
res = 50 # resolution
z = np.zeros((res, res))
for x in range(res):
for y in range(res):
z[x][y] = net.propagate([x/res, y/res])
plt.imshow(z, cmap=plt.cm.gray, interpolation='nearest')
plt.xlabel("input 1")
plt.ylabel("input 2")
plt.title("Output Activation")
plt.show()
Handwriting Categorization¶
First, I'll use the metakernel's %download magic to get the MNIST hand written data:
! pip install metakernel --user -U
import metakernel
metakernel.register_ipython_magics()
%download http://deeplearning.net/data/mnist/mnist.pkl.gz
Unzip the file:
!gunzip mnist.pkl.gz
From http://deeplearning.net/tutorial/gettingstarted.html we see that:
The pickled file represents a tuple of 3 lists: the training set, the validation set and the testing set. Each of the three lists is a pair formed from a list of images and a list of class labels for each of the images. An image is represented as numpy 1-dimensional array of 784 (28 x 28) float values between 0 and 1 (0 stands for black, 1 for white). The labels are numbers between 0 and 9 indicating which digit the image represents.
We read the Python2 pickled data:
import pickle
import gzip
import numpy
with open('mnist.pkl', 'rb') as f:
u = pickle._Unpickler(f)
u.encoding = 'latin1'
data = u.load()
train_set, validation_set, test_set = data
len(train_set)
len(train_set[0])
len(train_set[0][0])
net = Network(784, 100, 1)
inputs = [train_set[0][i] for i in range(len(train_set[0]))]
targets = [[train_set[1][i]/9.0] for i in range(len(train_set[0]))]
inputs = inputs[:100]
targets = targets[:100]
def display_digit(vector):
for r in range(28):
for c in range(28):
v = int(vector[r * 28 + c] * 10)
ch = " .23456789"[v]
print(ch, end="")
print()
net.display_test_input = display_digit
net.set_inputs(list(zip(inputs, targets)))
net.test(stop=2)
net.train(report_rate=10, tolerance=0.05)
net.test(stop=2)
for i in range(100):
output = net.propagate(inputs[i])
target = int(targets[i][0] * 9)
print("target:", target, "output:", output, "correct?", int(output * 10) == target)
h = net.get_history()
epochs = [x[0] for x in h]
tss = [x[1] for x in h]
percent = [x[2] for x in h]
import matplotlib.pyplot as plt
plt.plot(epochs, tss)
plt.xlabel("Epochs")
plt.ylabel("TSS Error")
plt.show()
plt.plot(epochs, percent)
plt.xlabel("Epochs")
plt.ylabel("Percentage Correct")
plt.show()
